Classification method and electronic apparatus

ABSTRACT

The disclosure provides a classification method and an electronic apparatus. The classification method includes the following steps. First feature data of multiple pictures of assembly is extracted, and each picture of assembly includes an operator at a station. The first feature data is converted into a first feature vector. Second feature data recording personal data of the operator is converted into a second feature vector. The first feature vector and the second feature vector are merged into a first feature matrix. The efficiency of the operator operating at the station is classified according to the first feature matrix to obtain a classification result.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Taiwan applicationserial no. 110114851, filed on Apr. 26, 2021. The entirety of theabove-mentioned patent application is hereby incorporated by referenceherein and made a part of this specification.

BACKGROUND Technical Field

The disclosure relates to a classification method and an apparatus, andmore particularly to a classification method and an electronic apparatusfor operator assignment.

Description of Related Art

In production line management, station assignment for operators affectsproduction capacity. Generally speaking, most operator assignmentmethods leverage historical data and design distribution logic accordingto business characteristics but cannot automatically update thedistribution logic. Therefore, if the business changes or the time andspace background changes, the original operator assignment methods willnot be able to work effectively and affect production capacity. Inaddition, operator assignment methods established with machine learningmodels in the past require collection of correctly labeled data foranalysis and training and thus tend to be limited by the number ofsamples when being used. Moreover, it is necessary that training viamachine learning models rely on historical data to build effectiveprediction features for accurate prediction results. In light of this,the past operator assignment methods overly rely on structured data andare prone to cause problems such as inaccurate predictions or lowrelevance of features. In terms of method expansion, if traditionalmachine learning is to be applied to other scenarios, retraining anothermachine learning model is inevitable. Therefore, in the face of slightchanges in prediction targets, it is necessary to design features andalgorithm again, which makes it impossible to import machine learningmodels efficiently and rapidly.

SUMMARY

The disclosure provides a classification method and an electronicapparatus, which may effectively predict working efficiency of eachoperator at each station.

The classification method of the disclosure includes the followingsteps. First feature data of multiple pictures of assembly is extracted,and each picture of assembly includes an operator at a station. Thefirst feature data is converted into a first feature vector. Secondfeature data recording personal data of the operator is converted into asecond feature vector. The first feature vector and the second featurevector are merged into a first feature matrix. The efficiency of theoperator operating at the station is classified according to the firstfeature matrix to obtain a classification result.

The electronic apparatus for operator classification of the disclosureincludes a storage and a processor. The storage stores at least one codefragment, multiple pictures of assembly, and personal data of anoperator. The processor is coupled to the storage and is configured toexecute the at least one code fragment to implement the classificationmethod.

Based on the above, the disclosure may solve the dilemma that it isdifficult to find effective features with only structured data in thepast, and may make predictions according to a variety of data, therebyimproving accuracy of final results.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an electronic apparatus according to anembodiment of the disclosure.

FIG. 2 is a structural diagram of a classification module according toan embodiment of the disclosure.

FIG. 3 is a flowchart of a classification method according to anembodiment of the disclosure.

FIG. 4 is a structural diagram of a classification module according toan embodiment of the disclosure.

FIG. 5 is a structural diagram of a classification module according toan embodiment of the disclosure.

DESCRIPTION OF THE EMBODIMENTS

FIG. 1 is a block diagram of an electronic apparatus according to anembodiment of the disclosure. With reference to FIG. 1, an electronicapparatus 100 includes a processor 110 and a storage 120. The processor110 is coupled to the storage 120.

The processor 110 is, for example, a central processing unit (CPU), aphysics processing unit (PPU), a programmable microprocessor, anembedded control chip, a digit signal processor (DSP), an applicationspecific integrated circuit (ASIC), or other similar apparatuses.

The storage 120 is, for example, any type of fixed or mobile randomaccess memory (RAM), read-only memory (ROM), flash memory, hard disk,other similar apparatuses, or a combination of these apparatuses. Thestorage 120 stores multiple code fragments, and the code fragments areexecuted by the processor 110 after installed. For example, the storage120 includes a classification module 121. The classification module 121is composed of one or more code fragments to be executed by theprocessor 110 for implementing the classification method.

The storage 120 further includes a database 122. The database 122 storesmultiple yield data and multiple historical data of multiple operatorsat the same station or at different stations corresponding to differentdates. These historical data include respective personal data of theoperators and historical pictures taken when the operators areperforming assembly operations at the stations. The processor 110clusters the operators according to the yield data stored in thedatabase 122 and then gives each operator an efficiency label (such as“good”, “middle”, or “bad”) at different time points. Afterwards, theefficiency labels of the operators corresponding to different dates andthe historical data of the operators are used to train theclassification module 121, and then the classification module 121 isused for subsequent predictions.

For example, according to multiple yield data, the processor 110 maycluster the corresponding operators into multiple groups via K-meansalgorithm, with each group corresponding to one efficiency label. Here,assuming that the number of the efficiency labels is 3, including“good”, “middle”, and “bad”, the K value of the K-means algorithm is setto 3 for clustering, such that a group of people with high failure ratesis defined as “bad”, a group of people with middle failure rates isdefined as “middle”, and a group of people with low failure rates isdefined as “good”. In this way, the efficiency label of each operator atone station per day is marked as shown in Table 1. Table 1 records thepersonal data and the efficiency label at a certain station on a certainday of each operator.

TABLE 1 operator recording date gender age . . . efficiency label A . .. female 25 . . . good B . . . male 24 . . . middle C . . . male 30 . .. bad . . . . . . . . . . . . . . . . . .

After the efficiency label of each operator is marked, the historicaldata corresponding to each operator may be further used to train theclassification module 121. For example, FIG. 2 is a structural diagramof a classification module according to an embodiment of the disclosure.In this embodiment, the classification module 121 uses a featureextractor 210 to extract features from multiple pictures of assembly Mand integrate the features into a first feature data F1. Here, thesepictures of assembly M are the historical pictures stored in the storage120. These pictures of assembly M are obtained by recording an imagestream when the operator A is performing assembly operation at a stationand then extracting multiple continuous image frames in the image streamas the pictures of assembly M. In some other embodiments, the picturesof assembly M may also be pictures taken at different time points whenthe operator A is performing assembly operation at the station (forexample, before assembly, during assembly, and after assembly). Thepurpose of using the pictures of assembly M to extract features andintegrating the features into the first feature data F1 lies in thatthese pictures of assembly M may represent some assembly habits,assembly postures, assembly sequence, or assembly efficiency of theoperator A during assembly.

Here, it is assumed that multiple pictures of assembly M and personaldata D of the operator A marked with one efficiency label as shown inTable 1 are used as input data. Here, the pictures of assembly M and thepersonal data D are stored in the storage 120. The personal data Dincludes gender, seniority, accommodation, eyesight, production linestation, recording date, age, assembly yield rate, and the like. Secondfeature data F2 is obtained after the personal data D is digitized.After the first feature data F1 and the second feature data F2 areobtained, feature conversion is performed respectively on the firstfeature data F1 and the second feature data F2 by a first featureconversion module 220 and a second feature conversion module 230. Then,a first merging module 240 performs a merge operation and inputs themerged result to a first classifier 250 to obtain scores correspondingto multiple efficiency labels, and the efficiency label corresponding tothe highest score is used as the final classification result. Infollowing, the final classification result is compared with thepre-marked efficiency labels to adjust parameters and/or weights in thefirst classifier 250 of the classification module 121.

FIG. 3 is a flowchart of a classification method according to anembodiment of the disclosure. With reference to FIG. 3, in step S305,the first feature data F1 of multiple pictures of assembly M isextracted. Here, the feature extractor 210 is used to obtain the firstfeature data F1 from the pictures of assembly M. The feature extractor210 may be implemented by a trained autoencoder. For example, multiplepictures of assembly of each operator are input for restoring andreconstructing graphics via deep learning to train the autoencoder, andfeatures of the input pictures of assembly for reconstruction areextracted as the first feature data. In addition, the feature extractor210 may also include a feature detector of a convolutional neuralnetwork (CNN). Generally speaking, a convolutional layer of the CNN mayobtain a feature map by the feature detector (also referred to as aconvolutional kernel or a filter).

Next, in step S310, the first feature data F1 is converted into a firstfeature vector. For example, the first feature conversion module 220includes a flatten function, a fully connected (FC) function, and anactivation function. First, the first feature data F1 is flattened withthe flatten function, which means the multi-dimensional first featuredata F1 is transformed into a one-dimensional matrix. Next, theone-dimensional matrix is input to the fully connected function toobtain a feature vector by adjusting the weight and deviation. Infollowing, the feature vector is input to the activation function toobtain the first feature vector.

Here, the fully connected function refers to the processing method of afully connected layer of a neural network. The fully connected functionis, for example, matrix multiplication, which is equivalent tofeature-space transformation. For example, affine transformation isperformed on input data by using the fully connected function tolinearly transform one vector space to another vector space, extractingand integrating useful information. The activation function may be arectified linear unit (ReLU) function, such as a ramp function, toenhance nonlinear characteristics.

In step S315, the second feature data F2 recording the personal data Dof an operator is converted into a second feature vector. For example,the second feature conversion module 230 includes the fully connectedfunction and the activation function. The fully connected function isused to linearly transform one vector space to another vector space, andthen the activation function (such as the ReLU function) is used toenhance the nonlinear characteristics.

Afterwards, in step S320, the first feature vector and the secondfeature vector are merged into a first feature matrix. Here, the firstmerge module 240 includes a concat function, the fully connectedfunction, and the activation function. The concat function is used tomerge the first feature vector and the second feature vector into onematrix, and then the fully connected function is used to linearlytransform one vector space of the matrix to another vector space, andthe activation function (such as the ReLU function) is further used toenhance the nonlinear characteristics. In this way, the first featurematrix is obtained.

In following, in step S325, the efficiency of the operator operating ata station is classified according to the first feature matrix to obtaina classification result. Specifically, the first feature matrix is inputto the first classifier 250 to obtain the classification result. Thefirst classifier 250 obtains scores of the operator corresponding tomultiple efficiency labels according to the input first feature matrix.Here, the first classifier 250 may be implemented by the fully connectedfunction. In this embodiment, multiple fully connected layers (the fullyconnected functions) are used, in which the last fully connected layerserves as a classifier and the other fully connected layers serve toextract features. Next, the processor 110 compares the classificationresult of the first classifier 250 with the pre-marked efficiency labelsto adjust parameters and/or weights in the first classifier 250 of theclassification module 121. After the training is completed, datacorresponding to unlabeled operators may be classified by using thefirst classifier 250.

In another embodiment, the classification module 121 may further beimplemented by two different feature extractors and two differentclassifiers as exemplified hereinafter.

FIG. 4 is a structural diagram of a classification module according toan embodiment of the disclosure. In this embodiment, the classificationmodule 121 uses two analysis modules 41 and 42. The analysis module 41includes an autoencoder 410, the first feature conversion module 220,the first merge module 240, and the first classifier 250. The analysismodule 42 includes a feature detector 420, a third feature conversionmodule 421, a second merge module 423, and a second classifier 425.Other elements with the same reference numbers represent elements havingthe same effects. Here, the autoencoder 410 needs to be trained inadvance before used, and the feature detector 420 may adjust itsinternal parameters during the training process of the first classifier250 of the classification module 121.

In the analysis module 41, the first feature data F1 is extracted frommultiple pictures of assembly M corresponding to an operator marked withthe efficiency label by the autoencoder 410. After the first featuredata F1 is obtained, the first feature data F1 is converted into thefirst feature vector by the first feature conversion module 220. Infollowing, the first feature vector and the second feature vector aremerged by the first merge module 240 to obtain the first feature matrix,and the merged first feature matrix is input to the first classifier 250to obtain first scores corresponding to multiple efficiency labels. Inother words, one efficiency label has one corresponding first score.

On the other hand, in the analysis module 42, third feature data F3 isextracted from multiple pictures of assembly M corresponding to anoperator marked with the efficiency label by the feature detector 420.After the third feature data F3 is obtained, the third feature data F3is converted into a third feature vector by the third feature conversionmodule 421. Here, the third feature conversion module 421 is similar tothe first feature conversion module 220. In following, the third featurevector and the second feature vector are merged by the second mergemodule 423 to obtain a second feature matrix, and the merged secondfeature matrix is input to the second classifier 425 to obtain secondscores corresponding to multiple efficiency labels. In other words, oneefficiency label has one corresponding second score.

Finally, the final classification result 440 is obtained based on thefirst scores obtained by the first classifier 250 and the second scoresobtained by the second classifier 425. For example, the classificationresult 440 records that the efficiency label of “good” scores 0.7, theefficiency label of “middle” scores 0.2, and the efficiency label of“bad” scores 0.1.

For example, Table 2 shows the first scores and the second scorescorresponding to different efficiency labels and weights correspondingthereto. The size of the weight may be determined based on theimportance of the feature extractor. The higher the importance, thegreater the weight.

TABLE 2 efficiency label first score second score good A1 B1 middle A2B2 bad A3 B3 weight W1 W2

For example, a composite score for an efficiency label of “good” isS_(good)=A1×W1+B1×W2, a comprehensive score for an efficiency label of“middle” is S_(middle)=A2×W1+B2×W2, and a comprehensive score for anefficiency label of “bad” is S_(bad)=A3×W1+B3×W2. Afterwards, theefficiency label corresponding to the highest one among thecomprehensive scores is taken as the final classification result.

In other embodiments, three or more feature extractors, featureconversion modules, merge modules, and classifiers may further be used.

FIG. 5 is a structural diagram of a classification module according toan embodiment of the disclosure. With reference to FIG. 5, thisembodiment is roughly the same as the structure shown in FIG. 4, and thedifference lies in that the structure shown in FIG. 5 further includes atiming module 510 and an operator assignment module 520. In thisembodiment, a specified time range for predicting work efficiency of anoperator may further be set, such that changes in the personal data Dmay be predicted within the specified time range by the timing module510, and the second feature data F2 may be extracted based on thepredicted personal data D. For example, assuming that the specified timerange is 90 days in the future, changes in the personal data D for eachday in the next 90 days is predicted by the timing module 510, such aswhether the operator is still working, whether the operator continues tolive in accommodation, or the like. In following, the classificationresults of the operator in each station for each day in the next 90 days(scores for the efficiency labels of “good”, “middle”, and “bad”) arepredicted the classification module 121. After that, operators areassigned by the operator assignment module 520 for the best personnelassignment. The operator assignment module 520 uses linear programmingalgorithm to assign each operator to the best station.

For example, a classification result of an operator U₀₁ at a stationSta₀₁ for each day in the next 90 days is predicted by theclassification module 121, and an efficiency score is given based on theclassification result. For example, the efficiency scores correspondingto the efficiency labels “good”, “middle”, and “bad” are 0, 1, and 2,respectively. As shown in Table 3, assuming that the efficiency score is0 as the classification result on Day 1 is “good”, the efficiency scoreis 2 as the classification result on Day 2 is “bad”, the efficiencyscore is 1 as the classification result on Day 3 is “middle”, and so on,then the efficiency score corresponding to the classification result onDay 90 is 1. Next, the predicted results for 90 days are summed up toobtain a prediction score as Score(Sta₀₁, U₀₁).

TABLE 3 station Sta₀₁; operator U₀₁ day in the future classificationresult efficiency score Day 1 good 0 Day 2 Bad 2 Day 3 middle 1 . . . .. . . . .  Day 90 middle 1 prediction score Score(Sta₀₁, U₀₁) = 0 + 1 +2 . . . + 1

In the same way, the prediction scores of multiple operators at multipledifferent stations are calculated by the classification module 121.After the predicted score of each operator at each station is obtained,operators may be further assigned according to the numbers of requiredpeople at these stations with the lowest assignment score obtained bysumming up the predicted scores of these operators after assignment.

For example, assuming that the total number of required people for thestation Sta₀₁ and a station Sta₀₂ is 4, and the total number of requiredpeople for a station Sta₀₃ and a station Sta₀₄ is 2, then the stationSta₀₁ currently has two positions that may be assigned, the stationSta₀₂ currently has six positions that may be assigned, the stationSta₀₃ currently has eight positions that may be assigned, and thestation Sta₀₄ currently has three positions that may be assigned.

Table 4 below shows which station each operator is to be assigned to,and values of X₁ to X_(i) are 1 or 0. A value of 1 means an operator isassigned to the corresponding station, while a value of 0 means anoperator is not assigned to the corresponding station. Taking theoperator U₀₁ for description, when the operator U₀₁ is assigned to thestation Stan, the value of X₁ is 1, and the values of X₂, X₃, and X₄ areall 0. In other words, when the value of one of X₁, X₂, X₃, and X₄ is 1,the values of the other three are all 0.

TABLE 4 operator Sta01 Sta02 Sta03 Sta04 U₀₁ X₁ X₂ X₃ X₄ U₀₂ X₅ X₆ X₇ X₈U₀₃ X₉ X₁₀ X₁₁ X₁₂ U_(j−1) X_(i−7) X_(i−6) X_(i−5) X_(i−4) U_(j) X_(i−3)X_(i−2) X_(i−1) X_(i)

Based on the positions currently available for assigned and the numberof required people at each station, the following conditions are set:

SUM₀₁ =X ₁ +X ₅ +X ₉ + . . . +X _(i-7) +X _(i-3)≤2;

SUM₀₂ =X ₂ +X ₆ +X ₁₀ + . . . +X _(i-6) +X _(i-2)≤6;

SUM₀₃ =X ₃ +X ₇ +X ₁₁ + . . . +X _(i-5) +X _(i-1)≤8;

SUM₀₄ =X ₄ +X ₈ +X ₁₂ + . . . +X _(i-1) +X _(i)≤3;

SUM₀₁+SUM₀₂=4;

SUM₀₃+SUM₁₀₄=2.

Among the above, SUM₀₁ represents the total number of people assigned atthe station Sta₀₁, SUM₀₂ represents the total number of people assignedat the station Stain, SUM₀₃ represents the total number of peopleassigned at the station Sta₀₃, and SUM₀₄ represents the total number ofpeople assigned at the station Sta₀₄. Based on the respective positionscurrently available for assignment at the stations Sta₀₁ to Sta₀₄, it isset that SUM₀₁≤2, SUM₀₂≤6, SUM₀₃≤8, and SUM₀₄≤3. In addition, based onthe numbers of required people (the total number of required people forthe station Sta₀₁ and the station Sta₀₂ is 4, and the total number ofrequired people for the station Sta03 and the station Sta04 is 2), it isset that SUM₀₁+SUM₀₂=4 and SUM₀₃+SUM₀₄=2.

With the above conditions, the predicted score of each operator at eachstation is used to obtain the best personnel assignment. In other words,the smaller the assignment score finally obtained, the better the effectthat may be obtained.

In addition, if data of new operators at new stations in the productionline appear in the future, there is no need to use all the data toretrain the entire model architecture, and the pros and cons of eachoperator at a new station may be predicted merely with a small amount ofdata at each station provided for updating and training the model.

To sum up, the disclosure may cluster historical data based on the prosand cons thereof, give each piece of the historical data an efficiencylabel, extract features in pictures of assembly of each operator by thefeature extractor, and integrate the features with past historical datato solve the dilemma that it is difficult to find effective featureswith only structured data in the past. In addition, after training, theresults of multiple models are integrated for predictions to distinguishthe pros and cons of each operator, and finally through the linearprogramming method, outstanding operators are prioritized to be assignedto suitable stations.

What is claimed is:
 1. A classification method, comprising: extractingfirst feature data of one of a plurality of pictures of assembly,wherein each of the plurality of pictures of assembly comprises anoperator located at a station; converting the first feature data into afirst feature vector; converting second feature data recording personaldata of the operator into a second feature vector; merging the firstfeature vector and the second feature vector into a first featurematrix; and classifying efficiency of the operator operating at thestation according to the first feature matrix to obtain a classificationresult.
 2. The classification method according to claim 1, before thestep of extracting the first feature data of the plurality of picturesof assembly, further comprising: recording an image stream duringassembly operation of the operator at the station; and extracting aplurality of continuous image frames in the image stream as theplurality of pictures of assembly.
 3. The classification methodaccording to claim 1, wherein the step of extracting the first featuredata corresponding to the plurality of pictures of assembly comprises:extracting the first feature data corresponding to the plurality ofpictures of assembly by a feature extractor, wherein the featureextractor comprises an autoencoder or a feature detector of aconvolutional neural network.
 4. The classification method according toclaim 1, after classifying the efficiency of the operator operating atthe station, the classification method further comprising: obtaining aplurality of first scores of the operator corresponding to a pluralityof efficiency labels; and determining one with the highest first scoreamong the efficiency labels as the classification result.
 5. Theclassification method according to claim 4, further comprising:extracting third feature data of one of the plurality of pictures ofassembly, wherein the first feature data is different from the thirdfeature data; converting the third feature data into a third featurevector; merging the third feature vector and the second feature vectorinto a second feature matrix; classifying the plurality of efficiencylabels of the operator according to the second feature matrix to obtaina plurality of second scores; and obtaining the classification resultcorresponding to the efficiency of the operator operating at the stationaccording to the plurality of first scores and the plurality of secondscores.
 6. The classification method according to claim 1, wherein thestep of converting the first feature data into the first feature vectorcomprises: converting the first feature data into the first featurevector by a flatten function, a fully connected function, and anactivation function.
 7. The classification method according to claim 1,wherein the step of classifying the efficiency of the operator operatingat the station comprises: classifying the efficiency according to thefirst feature matrix by a classifier; wherein the classification methodfurther comprising: clustering a plurality of yield data of a pluralityof operators at the station corresponding to different dates by aclustering algorithm to obtain an efficiency label of the plurality ofoperators corresponding to each time point; and training the classifierby the efficiency label corresponding to each of the plurality ofoperators at each time point and a plurality of historical data of eachof the plurality of operators, wherein each of the plurality ofhistorical data comprises the personal data of each of the plurality ofoperators and historical pictures taken during assembly operation at thestation.
 8. The classification method according to claim 1, wherein thepersonal data comprises gender, seniority, accommodation, eyesight,production line station, recording date, age, assembly yield, or anycombination thereof.
 9. The classification method according to claim 1,before the step of converting the second feature data recording thepersonal data of the operator into the second feature vector, furthercomprising: predicting change in the personal data within a specifiedtime range by a timing module, and determining the personal data basedon prediction as the second feature data.
 10. An electronic apparatusfor operator classification, comprising: a storage, storing at least onecode fragment, a plurality of pictures of assembly, and personal data ofan operator; and a processor, coupled to the storage and configured toexecute the at least one code fragment to implement: extracting firstfeature data of one of the plurality of pictures of assembly, whereineach of the plurality of pictures of assembly comprises the operatorlocated at a station; converting the first feature data into a firstfeature vector; converting second feature data recording the personaldata into a second feature vector; merging the first feature vector andthe second feature vector into a first feature matrix; and classifyingefficiency of the operator operating at the station according to thefirst feature matrix to obtain a classification result.